Cost Complexity of Proactive Learning via a Reduction to Realizable Active Learning

Proactive Learning is a generalized form of active learning with multiple oracles exhibiting different reliabilities (label noise) and costs. We propose a general approach for Proactive Learning that explicitly addresses the cost vs. reliability tradeoff for oracle and instance selection. We formulate the problem in the PAC learning framework with bounded noise, and transform it into realizable active learning via a reduction technique, while keeping the overall query cost small. We propose two types of sequential hypothesis tests (denoted as SeqHT) that estimate the label of a given query from the noisy replies of different oracles with varying reliabilities and costs. We prove correctness and derive cost complexity of the proposed algorithms. This research is supported by grants from the National Science Foundation.